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INTRODUCTION 

Information  overload  can  quickly  overwhelm  users  of 
current  Navy  displays.  Too  often  users  must  perform  tedious 
search  tasks  and  mental  operations  when  processing 
displayed  information;  factors  which  may  reduce  the 
effectiveness  of  the  combat  system.  New  display  technolo¬ 
gies  such  as  color  and  graphics  offer  the  potential  for 
decreasing  the  perceptual  and  cognitive  burden  on  users  ol 
current  status  displays.  Conversely,  these  technologies  can 
also  contribute  to  information  overload  if  improperly  used. 
There  is  a  need  to  determine  the  most  effective  use  of  new 
display  technologies  if  we  are  to  contribute  to  belter  user- 
system  performance. 

The  objective  of  this  study  was  to  compare  the  relative 
effectiveness  of  two  experimental  formats  lor  presenting 
missile  status  data  to  a  current  formal  of  displaying  such  data 
aboard  U.S.  Navy  ships. 

METHOD 

Participants 

Sixteen  male  U.S.  Navy  personnel  (15  enlisted  and  one 
officer)  participated  in  the  study.  The  participants  had  served 
an  average  of  9.5  years  in  the  Navy  and  had  an  average  of  52 
months  Combat  Information  Center  (CIC)  experience.  They 
ranged  in  age  from  25  to  38  years,  with  a  mean  of  31 .5  years 
and  standard  deviation  of  4.6  years.  All  participants  reported 
having  normal  or  corrected-to-normal  vision. 

Apparatus 

A  Macintosh  Model  tlci  computer  was  used  to  present  the 
task  materials  and  to  record  lest  participants'  responses  and 
reaction  limes.  Task  materials  were  displayed  on  a  19-In. 
diagonal  high-resolution  color  monitor.  A  numeric  keypad 
with  special  keys  configured  for  yes  and  no  responses 
served  as  the  Input  device,  The  monitor  and  keypad  were 
Installed  in  a  mock-up  ol  a  prototype  CIC  operator  console 
described  In  SPAWAR  (1991), 


Three  missile  status  data  presentation  formats  were 
tested;  character  readout  (CRO),  text-only,  and  text- 
graphics.  The  CRO  format  (see  Figure  1)  was  an  exact  replica 
of  a  current  CIC  display  format.  In  addition  to  changes  In  color 
and  layout,  the  experimental  formats  (see  Figure  2)  provide 
the  following  design  changes  to  the  current  displays: 


1 .  Missile  inventory  data  is  segregated  for  A-  and  B  rails  on 
the  FWD  (forward)  and  AFT  launchers,  and 


2 .  Row  and  column  totals  lor  different  missile  and  launchers 
are  displayed. 
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Figure  1 .  Example  of  the  CRO  format. 

Further,  launcher  and  system  status  fields  tor  the  experi¬ 
mental  formats  (shown  in  the  lower  panels  ol  Figure  2)  were 
reformatted  and  separated  from  missile  inventory  data. 
These  displays  also  make  use  of  fewer,  more  consistent 
terms  and  symbols  to  represent  operational  status  data  than 
the  CRO  format.  (Figure  3  lists  all  symbols  used  in  the 
experimental  lormats). 

Besides  differences  In  displaying  launcher  and  system 
status  data,  note  the  bar  charts  to  augment  numeric  data 
entries  in  Figure  2b.  Their  inclusion  stemmed  from  the 
following  Smith  &  Mosler  (1986)  guidelines: 

2,4.2  When  users  must  quickly  scan  and  compare  rotated 
sets  of  data,  consider  n  graphic  format  to  display  the 
data.  (p.  130) 

2.4,9  When  precise  reading  of  a  graphic  display  is 
required,  annotate  the  display  with  actual  data  values 
to  supplement  their  graphic  representation,  (p.  132) 
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Figure  2.  Example  of  the  text-only  lormat  (a)  and  the  text- 
graphics  format  (b) 


Another  ditference  between  presentation  lormats  con¬ 
cerned  the  use  ot  color.  All  text  and  numeric  characters  in 
the  CRO  format  appeared  in  phosphorescent  green  against 
a  uniform  black  background.  For  the  other  lormats,  values  of 
zero  in  the  missile  inventory  fields  appeared  in  red,  as  did 
symbols  representing  "Down"  in  the  operational  status  fields. 
Symbols  for  the  term  “Up  &  On”  appeared  in  green;  those  for 
the  term  “Degraded"  in  orange;  and  the  background  for  the 
“Test"  and  “Notest"  graphic  icons  in  yellow.  All  other  sym¬ 
bols  were  displayed  as  shown  in  Figures  2  &  3  with  the 
exception  of  the  background,  which  was  a  uniform  light  gray 
color. 
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rest  Materials 

Test  materials  consisted  of  two  parallel  versions  ot  33 
questions  each  The  questions,  modeled  alter  ihe  work  ol 
Hitt  el  al.  (1961),  were  divided  into  lour  categories  of  display 
reading  lasks:  count,  compare,  identity,  and  criterion 
Examples  ol  the  two  versions  of  test  questions  administered 
in  each  ot  the  lour  task  categories  are  listed  in  Table  i .  The 
order  of  question  administration  was  randomized  within  each 
test  version  and  presentation  format,  yeilding  six  unique 
question  orderings 


Table  1 .  Examples  ol  test  questions  administered  in  each 
of  lour  task  categories. 


Task 

Category 

No.  of 
Items 

Test 

Version 

Example  Questions 

Count 

7 

1 

What's  the  total  standard  missile 
inventory  AFT? 

2 

What's  the  total  standard  missile 
inventory  FWD? 

Compare 

11 

1 

Are  there  more  standard  missiles 
AFT  A-rail  than  AFT  B-rail? 

2 

Are  there  more  standard  missiles 
FWD  A-  rail  than  FWD  B-rail? 

Identity 

6 

1 

What's  the  SM2/I  missile  inventory 
in  the  AFT  A-rail? 

2 

What's  the  SM2/I  missile  inventory 
in  the  AFT  B-rail? 

Criterion 

9 

1 

Are  FWD  &  AFT  launchers  ASW 
enqaqement  capable? 

2 

Are  FWD  &  AFT  launchers  AAW 
enqaqement  capable? 

Procedure 

Each  participant  was  tested  individually  in  two  consec¬ 
utive  sessions  in  a  classroom  setting.  At  the  outset  of  the 
session,  instructions  were  read  aloud  to  provide  the 
participant  a  general  orientation  to  the  study,  alter  which  he 
completed  a  voluntary  consent  and  background  inlormation 
form.  On-screen  instructions  and  examples  of  the  stimulus 
materials  to  be  used  with  the  first  presentation  format  were 
then  shown.  For  the  text-only  and  text-graphics  formats,  this 
included  a  10-item  task  in  which  the  participant  associated 
text  or  graphic  symbols  with  their  proper  operational  terms. 
On-screen  feedback  was  provided  for  incorrect  responses  to 
the  association  task.  In  addition,  incorrectly  answered  items 
were  reinserted  (at  random)  into  the  queue  until  the 
participant  met  the  task  criterion  ol  100  percenl  response 
accuracy. 

Participants  then  performed  a  4-iiem  practice  exercise  to 
become  familiar  with  the  presentation  formal  to  be  used  and 
actions  to  be  performed.  On-screen  feedback  was  provided 
lor  incorrect  responses  to  Ihe  practice  exercises  until  the 
criterion  of  100  percent  response  accuracy  was  met.  Next, 
the  33  questions  contained  in  the  first  presentation  format 
were  administered,  followed  by  a  short  rest  break.  Identical 
procedures  were  used  when  administering  the  next  set  ol 
questions  in  Ihe  alternative  formal.  Alter  completing  the 
second  sossion,  participants  evaluated  the  adequacy  ol  the 
text  or  graphic  symbols  used  in  the  experimental  formats, 
then  completed  a  separate  user  preference  questionnaire. 

ExPBrlmenlal.HyRothesea-andJDesion 


Figure  3,  Symbols  used  In  the  text-only  format  (a)  and  toxl- 
graphics  format  (b) 


The  following  hypotheses  served  as  the  framework  lor 
comparing  task  performance  as  a  function  of  display 
presentation  format; 
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I  Participants  will  demonstrate  taster  response  times  and 
higher  accuracy  on  all  categories  ol  tasks  when  using 
tho  experimental  lormats  Ilian  when  using  the  CRO 
format 

II  Since  the  display  lormats  and  inlormation-processing 
requirements  lor  count  and  identity  tasks  are  quite 
similar,  no  performance  differences  will  be  lound 
between  the  text-only  and  text-graphics  formats  on 
these  tasks. 

III.  Because  graphics  allow  a  "quick  scan"  of  the  display  to 
obtain  initial  estimates  ol  high-low  values,  participants 
using  the  text-graphics  formal  will  perform  compare 
tasks  taster  than  participants  using  the  text-only  format. 

IV.  Because  graphic  icons  convey  the  same  inlormation  as 
text  in  a  less  "perceptually-cluttered"  tormat,  partici¬ 
pants  using  the  text-graphics  format  will  perform 
criterion  tasks  faster  and  more  accurately  than 
participants  using  the  text-only  format. 

A  mixed  factor  design  served  as  the  model  for  data 
analysis.  The  between-subjects  (actor  was  assignment  of 
test  participants  to  one  ol  the  two  experimental  formats.  The 
within-subjects  lactor  was  replications  ol  each  task  category 
within  alternative  presentation  formats:  CRO  vs.  text-only  and 
CRO  vs.  text-graphics.  Test  version  and  order  of  adminis¬ 
tering  the  two  presentation  formats  was  counterbalanced 
using  Latin  squares.  Dependent  measures  were  response 
speed  and  accuracy.  Response  speed  was  measured  from 
the  initial  presentation  of  a  question  to  an  entry  response. 
Accuracy  was  the  percentage  ol  questions  answered 
correctly  by  the  test  participant. 

RESULTS 

Preliminary  Analyses 

Two  separate  analysis  of  variance  (ANOVAs)  were 
conducted  preparatory  to  the  main  tests  of  the  experimental 
hypotheses.  The  tirst  tested  for  differences  in  performance 
speed  and  accuracy  as  a  function  ol  lest  version  and  display 
format  presentation  order;  the  second  as  a  function  ot  lest 
participant  age,  months  of  CIC  experience,  and  computer 
proficiency. 

Results  showed  nonsignificant  £-ratios  for  all  main  ef¬ 
fects  and  interactions.  Accordingly,  it  was  concluded:  (1)  the 
two  question  sets  are  equivalent,  (2)  order  of  administering 
the  presentation  formats  did  not  produce  any  asymmetrical 
transfer  effects,  and  (3)  the  participants'  background  char¬ 
acteristics  did  not  have  any  differential  effects  on  task  perfor¬ 
mance  speed  or  accuracy. 

Main  Analyses 

Hypothesis  I.  Because  the  four  categories  ol  tasks 
contained  unequal  numbers  of  questions,  relevant  data  (or 
this  hypothesis  were  analyzed  in  8  mixed  factor  ANOVAs— 
one  for  each  fask  category  separately  on  the  speed  and 
accuracy  measures.  Results,  summarized  In  Tables  2  &  3, 
show  the  experimental  formats  wore  superior  to  the  CRO  in 
terms  of  response  accuracy  and  speed  for  tho  categories  ol 
count  and  compare  tasks. 

Specifically,  examination  of  Table  2  shows  when  using 
the  experimental  formats,  participants  had  an  absolute 
difference  of  14  and  11  percent  higher  accuracy  on  count 
and  compare  tasks  than  when  using  the  CRO  formal.  This 
translates  Into  an  18  and  13  percent  relative  Improvement 


over  the  CRO  lor  these  task  j  Similarly  l.i’>  '.<•••  .* 
a  speed  advantage  tor  tho  expenriMjnt.ii  :  ••i.it:  !’••• 

CRO  participants  look  54  and  3b  jmm 
responding  to  count  and  compare  rtcrtiv  nf.p-clivcr 


Table  2  Summary  of  analysis  ol  variance  and  means  for 
response  accuracy 


Mean  Accuracy 
(in  percent) 

Category 

df 

F 

Exper. 

CRO 

Count 

1,  110 

9.02** 

92 

78 

Compare 

1,  174 

8.40** 

93 

82 

Identify 

1, 94 

0.22 

91 

89 

Criterion 

1,  142 

0.40 

96 

97 

“Significant  at  g<.0l 

Table  3.  Summary  ol  analysis  of  variance  and  means  for 
response  speed 


Mean  Speed 
(in  seconds) 

Category 

df 

F 

Exper. 

CRO 

Count 

1,  110 

18.31** 

7.64 

11.79 

Compare 

1,  174 

38.90** 

11.03 

15.18 

Identify 

1,94 

0.27 

8.04 

8.23 

Criterion 

1,  142 

0.0004 

8.41 

8.42 

“Significant  at  b<01 


Contrary  to  the  Hypothesis  I  predictions,  no  reliable 
differences  were  found  between  the  CRO  and  experimental 
formats  relative  to  performance  speed  or  accuracy  on  identify 
and  criterion  tasks. 

Hypothesis  II.  This  hypothesis  predicted  no  perfor¬ 
mance  differences  between  the  experimental  lormats  for 
count  and  identify  tasks.  Relevant  data  lor  this  hypothesis 
were  extracted  from  the  mixed  factor  ANOVAs  described 
above,  As  predicted,  no  significant  differences  were  tound 
between  the  text-only  and  text-graphics  lormats  relative  to 
performance  speed  or  accuracy  on  count  and  identify  tasks. 
The  means  tested  in  these  and  other  comparisons  are 
presented  in  Figures  4  &  5. 

Hypothesis  ill.  This  hypothesis  predicted  faster  re¬ 
sponse  times  for  the  text-graphics  formal  than  the  lext-only 
format  on  compare  tasks.  ANOVA  results  showed  no  reliable 
difference  betwoen  these  presentation  formats  in  terms  ot 
performance  speed  (E  (1 ,1 74)»  0.52,  g<.48);  thus  failing  to 
support  Hypothesis  III  predictions. 
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Figure  4.  Response  accuracy  lor  two  experimental 
presentation  formats  on  lour  tasks. 


*  Significant  atp<  .01 


Figure  5.  Response  speed  lor  two  experimental 
presentation  formats  on  four  tasks 

Hypothesis  IV.  This  hypothesis  predicted  faster,  more 
accurate  performance  for  the  text-graphics  format  than  the 
toxt-onty  format  on  criterion  tasks.  While  no  difference  was 
lound  between  these  lormals  relative  to  accuracy,  tho  main 
effect  for  performance  speed  did  roach  statistical 
significance:  £(1,142)  -  5.50,  n<-02'  Contrary  to  hypothesis 
predictions,  participants  who  used  the  text-only  formal  had 
(aster  response  times  (7.82  seconds)  than  participants  who 
used  the  text-graphics  format  (9.01  seconds). 


Participant  background  characteristics  and  experimental 
presentation  formats  were  also  included  ,t-.  factors  in 
analyses  ol  ratings  on  two  user  preference  surveys  In  the 
tirst  survey,  participants  rated  how  well  each  text  or  graphic 
symbol  (see  Figure  3)  represented  its  corresponding 
operational  term  Scale  values  lor  this  survey  ranged  from 
1— not  representative  to  5— highly  representative  In  the 
second  survey,  participants  rated  items  concerning  their 
attitudes  toward  the  experimental  lormals  on  a  dilterent  5- 
point  scale. 

No  reliable  differences  were  lound  in  ratings  on  either 
survey  as  a  function  of  participant  background  or  presen¬ 
tation  format  (actors.  Significant  deferences  were  tound, 
however,  between  ratings  ot  text  and  graphic  symbols  on  the 
first  survey.  Specilically.  average  ratings  lor  the  "Silent" 
symbol  were  lower  than  all  other  symbols  except  "Nolest" 
and  "Transmitting"  at  p  <.05 

Average  ratings  assigned  to  the  experimental  lormals  on 
the  second  survey  were  generally  quite  lavorable  (see 
Figure  6).  Participants  rated  these  lormats  as  “Somewhat 
better"  than  current  CIC  displays  (Mean  4.2).  Positive  ratings 
were  also  assigned  lo  items  concerning  the  participants' 
overall  attitude  toward  the  experimental  formats  (Mean  4.5), 
how  easy  they  were  to  learn  (Mean  4.8),  and  their  ease  ot 
use  (Mean  4.1). 


Figure  6.  Average  user  preference  ratings  for  experi¬ 
mental  presentation  lormats. 


CONCLUSIONS  AND  DESIGN  IMPLICATIONS 

While  results  show  the  experimental  lormals  ottered 
both  a  speed  and  accuracy  advantage  over  the  CRO  lor 
count  and  compare  tasks,  no  reliable  diltercnces  were  tound 
between  these  formats  for  identify  and  criterion  tasks. 

Several  factors  may  explain  these  findings.  First,  the 
layout  ol  tho  experimental  formats  clearly  reduced  the  time 
needed  lo  locate  and  process  pertinent  data— particularly  for 
Hems  Involving  row  and  column  totals.  These  layouts  were  ot 
little  benefit  lor  ktontify  and  criterion  tasks.  The  majority  ot 
those  tasks  involved  locating  discroio  values  within  a  single 
row  or  column. 

Second,  tho  CRO  format  may  havo  ottored  better 
stimufus-responso  compatibility  lor  certain  criterion  task  items 
than  the  experimental  formats.  In  several  cases,  YES  and  NO 
terms  used  to  doscribo  tho  operational  status  ot  missile 
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launchers  in  the  CRO  lormat  were  synonymous  with  correct 
responses  to  test  items.  The  experimental  lormals  in  con¬ 
trast.  required  text  or  graphic  symbols  to  be  translated  into 
operational  terms  prior  to  formulating  a  yes  or  no  response 

Contacting  results  obtained  for  comparisons  between 
the  experimental  formats  used  in  this  study  also  warrant 
discussion.  Specifically,  the  lack  of  a  speed  advantage  for 
the  text-graphics  format  on  compare  tasks  (Hypothesis  III), 
may  be  explained  in  two  ways.  First,  it  is  possible  that 
participants  simply  ignored  the  relative  height  of  the  bar- 
graphs  when  making  comparisons;  instead,  focusing  on  the 
displayed  numeric  values.  Second,  the  wording  of  these 
items  may  have  effectively  precluded  a  quick  scan  of  the 
display  to  obtain  estimates  of  high-low  values.  Perhaps  a 
more  appropriate  way  of  testing  for  a  speed  advantage  in  the 
text-graphics  format  would  involve  rewording  the  items  (e  g., 
Are  there  enough  SM2-II  missiles  FWD  to  engage  threat  £?). 
Here,  the  comparison  is  implicit — tapping  internalized  rules 
concerning  the  number  ol  missiles  required  to  engage  the 
threat.  When  displayed  in  the  text-graphics  format,  bar- 
graphs  may  then  function  as  a  type  of  "fuel  gauge."  allowing 
the  operator  to  tell  at  a  glance  if  sufficient  missiles  were 
available. 

Finally,  the  speed  advantage  lor  the  text-only  formal  on 
criterion  tasks  (Hypothesis  IV),  may  be  attributed  to  added 
information-processing  “overhead"  for  the  text-graphics 
lormat.  That  is,  participants  who  used  the  text-only  format 
may  have  spent  slightly  less  time  translating  displayed 
symbols  into  their  operationally-equivalent  terms  than 
participants  who  used  the  text-graphics  formal. 

Based  on  the  study  findings,  a  specific  recommendation 
for  improving  the  design  ol  current  Navy  status  displays  is  as 
follows: 

•  The  text-only  lormat,  exemplified  in  Figure  2a,  is  the 
best  choice  for  upgrading  the  display  of  missile  and 
system  status  data  from  the  current  CRO  format. 

In  addition,  the  following  guidelines  provide  possible 
explanations  of  these  results: 

1 .  To  facilitate  rapid  scanning  and  assimilation  of  multiple 
data  elements  in  tabular  form,  segregate  the  display  such 
that  only  the  information  which  is  deemed  relevant  to  the 
task  at  hand  is  perceived  and/or  acted  on  by  users. 

2.  Ensure  that  data  displayed  in  tabular  lorm  is  organized 
into  clearly  defined  and  labeled  rows  and  columns. 

3.  If  the  tabular  display  is  to  be  used  for  tasks  other  than 
locating  discrete  values  within  a  particular  row  or  column, 
provide  a  capability  for  generating  and  dynamically 
updating  totals— by  applicable  rows,  columns,  or 
aggregations  ol  rows  and  columns. 

4.  Ensure  that  terms  or  symbols  representing  operational 
status  or  state  data  are  directly  compatible,  and 
preferably,  synonymous  with  responses  required  of  the 
operator  when  processing  these  data. 
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